In this script, there is conducted the estimation for the measure_marginal approach for a single given env.

The env = nethermind, the programs file =pg_marginal_full5_c50_step5_shuffle.csv, the resutls file = results_nethermind_marginal_full5_c50_step5.csv.

Switch removed_outliers to FALSE to see the comparison.

if (!removed_outliers) {
  boxplot(measurements[which(measurements$env == env), 'measure_total_time_ns'] ~ measurements[which(measurements$env == env), 'opcode'], las=2, outline=TRUE, log='y', main=paste(env, 'all'))
}
if (removed_outliers) {
  measurements = remove_compare_outliers(measurements, 'measure_total_time_ns', c(env))
}

Every sample starts with a fresh evm instance. We investigate whether the results may depend on the time from evm start - related to run_id. To avoid being overrun by the number of images, all op_count for a given run_id are are placed, so values are not centered. That should not be an issue.

Now we can investigate the linear regressions.

for (opcode in all_opcodes) {
  estimate = compute_all(opcode=opcode, env=env, use_median=TRUE, plots=ifelse(details,'all','scatter'))
  estimates[nrow(estimates) + 1, ] = c(opcode, estimate, env)
}

estimates
##                 op estimate_marginal_ns estimate_marginal_ns_stderr        env
## 1              ADD           55.1498839                    6.067989 nethermind
## 2              MUL           98.2540926                    3.421171 nethermind
## 3              SUB           63.5008107                    8.203837 nethermind
## 4              DIV           50.8981713                    9.249762 nethermind
## 5             SDIV           78.8294636                   10.092430 nethermind
## 6              MOD           59.2657112                    7.018241 nethermind
## 7             SMOD           81.5359026                    5.829317 nethermind
## 8           ADDMOD           87.8504758                    7.057642 nethermind
## 9           MULMOD          320.1560816                    8.884945 nethermind
## 10             EXP          471.8929515                    9.391311 nethermind
## 11      SIGNEXTEND           52.0139813                    5.666228 nethermind
## 12              LT           27.1808411                    2.869118 nethermind
## 13              GT           26.5517954                    6.560945 nethermind
## 14             SLT           47.2020204                    4.875711 nethermind
## 15             SGT           43.3603060                    5.543385 nethermind
## 16              EQ           38.8250773                    5.726892 nethermind
## 17          ISZERO           28.1897472                    4.498050 nethermind
## 18             AND           14.0102186                    3.935388 nethermind
## 19              OR           12.9874043                    5.641236 nethermind
## 20             XOR           17.9449026                    3.629751 nethermind
## 21             NOT           20.5049865                    3.638308 nethermind
## 22            BYTE           40.8649200                    4.658759 nethermind
## 23             SHL           65.5772081                    3.076853 nethermind
## 24             SHR           70.3895714                    4.432958 nethermind
## 25             SAR           91.4873748                    6.703635 nethermind
## 26         ADDRESS           19.9003640                    3.146706 nethermind
## 27          ORIGIN           19.2789136                    2.028033 nethermind
## 28          CALLER           16.5534593                    3.410519 nethermind
## 29       CALLVALUE           20.9725125                    3.208303 nethermind
## 30    CALLDATALOAD           39.5309923                    5.374943 nethermind
## 31    CALLDATASIZE           24.0938422                    3.059577 nethermind
## 32    CALLDATACOPY          130.1881251                   11.999634 nethermind
## 33        CODESIZE           23.3357660                    3.395562 nethermind
## 34        CODECOPY          142.2419371                    4.093720 nethermind
## 35        GASPRICE           18.2804157                    4.195672 nethermind
## 36  RETURNDATASIZE           33.4928369                    3.478116 nethermind
## 37  RETURNDATACOPY          185.3728191                    5.076237 nethermind
## 38        COINBASE           26.7730210                    2.334185 nethermind
## 39       TIMESTAMP           21.2664547                    2.618775 nethermind
## 40          NUMBER           26.9096198                    2.239707 nethermind
## 41      DIFFICULTY           25.4834150                    3.904577 nethermind
## 42        GASLIMIT           26.9898463                    3.736113 nethermind
## 43         CHAINID           27.7680044                    1.665004 nethermind
## 44     SELFBALANCE           95.3616850                    3.434738 nethermind
## 45             POP           18.0013834                    2.626372 nethermind
## 46           MLOAD           71.3291818                    5.758019 nethermind
## 47          MSTORE           69.5988850                    3.550924 nethermind
## 48         MSTORE8           66.8125313                    5.221889 nethermind
## 49            JUMP           74.2215942                   15.665089 nethermind
## 50           JUMPI          122.5617731                   35.710821 nethermind
## 51              PC           13.9504306                    4.258192 nethermind
## 52           MSIZE           28.3753285                    3.506293 nethermind
## 53             GAS           24.2686287                    1.698879 nethermind
## 54        JUMPDEST           15.4979513                    2.803608 nethermind
## 55           PUSH1           20.6223639                    3.365546 nethermind
## 56           PUSH2           16.0281900                    1.687624 nethermind
## 57           PUSH3           14.8542623                    2.810397 nethermind
## 58           PUSH4           17.6676890                    3.959537 nethermind
## 59           PUSH5           15.4905842                    2.134714 nethermind
## 60           PUSH6           21.2779596                    2.733717 nethermind
## 61           PUSH7           12.5924331                    2.393437 nethermind
## 62           PUSH8           18.1525101                    2.610391 nethermind
## 63           PUSH9           16.8521547                    2.933991 nethermind
## 64          PUSH10           14.4487671                    3.084651 nethermind
## 65          PUSH11           16.9585193                    1.200009 nethermind
## 66          PUSH12           19.6999856                    3.307883 nethermind
## 67          PUSH13           12.7743463                    2.569607 nethermind
## 68          PUSH14           18.8540378                    2.751063 nethermind
## 69          PUSH15           17.1800597                    2.354584 nethermind
## 70          PUSH16           13.2561839                    1.980957 nethermind
## 71          PUSH17           16.3043976                    3.152565 nethermind
## 72          PUSH18           20.6271595                    2.149994 nethermind
## 73          PUSH19           16.4115656                    3.775739 nethermind
## 74          PUSH20           11.7268314                    4.177968 nethermind
## 75          PUSH21           11.5053332                    4.032250 nethermind
## 76          PUSH22           15.7076332                    1.590536 nethermind
## 77          PUSH23           15.8108294                    2.739323 nethermind
## 78          PUSH24           20.8675434                    3.137074 nethermind
## 79          PUSH25           17.7450269                    4.221478 nethermind
## 80          PUSH26           19.0068420                    2.770996 nethermind
## 81          PUSH27           17.4338244                    2.857109 nethermind
## 82          PUSH28           19.7821651                    2.487930 nethermind
## 83          PUSH29           20.0163536                    2.395379 nethermind
## 84          PUSH30           16.4243487                    3.695990 nethermind
## 85          PUSH31           21.2677979                    4.990745 nethermind
## 86          PUSH32           12.8241524                    2.434049 nethermind
## 87            DUP1           11.7421393                    6.772931 nethermind
## 88            DUP2           10.2061600                    4.104871 nethermind
## 89            DUP3           20.5334039                    3.230047 nethermind
## 90            DUP4            6.2445510                    3.800998 nethermind
## 91            DUP5           14.3238132                    4.656438 nethermind
## 92            DUP6            7.9439303                    2.277231 nethermind
## 93            DUP7           15.9011144                    3.617586 nethermind
## 94            DUP8           15.7369582                    5.464432 nethermind
## 95            DUP9           19.3663976                    4.477369 nethermind
## 96           DUP10            7.7553369                    4.465901 nethermind
## 97           DUP11           11.1566673                    3.365875 nethermind
## 98           DUP12           14.9211012                    5.883714 nethermind
## 99           DUP13           13.2647338                    3.147786 nethermind
## 100          DUP14           15.8084483                    2.051078 nethermind
## 101          DUP15           23.0423156                    4.082010 nethermind
## 102          DUP16            4.8660437                    3.863275 nethermind
## 103          SWAP1           10.9318200                    3.624123 nethermind
## 104          SWAP2           14.8677354                    3.544437 nethermind
## 105          SWAP3           11.5113578                    3.330087 nethermind
## 106          SWAP4           19.2324492                    2.798889 nethermind
## 107          SWAP5           16.9412503                    4.747053 nethermind
## 108          SWAP6           11.2875922                    2.839656 nethermind
## 109          SWAP7           12.9346662                    5.652068 nethermind
## 110          SWAP8            7.5718778                    4.411163 nethermind
## 111          SWAP9           10.4754353                    2.848118 nethermind
## 112         SWAP10           10.7074240                    3.359298 nethermind
## 113         SWAP11           15.8490828                    4.864963 nethermind
## 114         SWAP12           12.3999309                    2.291482 nethermind
## 115         SWAP13           14.4696544                    4.324026 nethermind
## 116         SWAP14           16.0388840                    2.695125 nethermind
## 117         SWAP15           16.4337392                    3.850519 nethermind
## 118         SWAP16           14.0107249                    3.213835 nethermind
## 119           SHA3          941.0087559                   31.602223 nethermind
## 120           LOG0          203.2269547                    8.393729 nethermind
## 121           LOG1          221.6560893                   10.494647 nethermind
## 122           LOG2          255.8178740                   11.975905 nethermind
## 123           LOG3          287.0249619                   16.168189 nethermind
## 124    EXTCODEHASH          282.7897364                    5.313746 nethermind
## 125    EXTCODESIZE          311.2001161                    7.756004 nethermind
## 126    EXTCODECOPY          530.7251280                    7.923440 nethermind
## 127         CREATE         6704.6370217                   47.952914 nethermind
## 128           CALL         2641.6708993                   34.951724 nethermind
## 129     STATICCALL         2657.9683940                   37.294994 nethermind
## 130   DELEGATECALL         2655.1748644                   37.647686 nethermind
## 131         RETURN         1478.1269649                   16.203495 nethermind
## 132         REVERT           -2.6223386                    6.808934 nethermind
## 133           STOP            0.4707091                    7.711025 nethermind

The results are exported to ./nethermind_marginal_estimated_cost.csv.